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Abstract 

This  paper  describes  a  novel  approach 
to  inducing  lexico-structural  transfer 
rules  from  parsed  bi-texts  using  syn¬ 
tactic  pattern  matching,  statistical  co¬ 
occurrence  and  error-driven  filtering. 

We  present  initial  evaluation  results  and 
discuss  future  directions. 

1  Introduction 

This  paper  describes  a  novel  approach  to  inducing 
transfer  rules  from  syntactic  parses  of  bi-texts  and 
available  bilingual  dictionaries.  The  approach 
consists  of  inducing  transfer  rules  using  the  four 
major  steps  described  in  more  detail  below:  (i) 
aligning  the  nodes  of  the  parses;  (ii)  generating 
candidate  rules  from  these  alignments;  (iii)  order¬ 
ing  candidate  rules  by  co-  occurrence;  and  (iv)  ap¬ 
plying  error-driven  filtering  to  select  the  final  set 
of  rules. 

Our  approach  is  based  on  lexico-structural 
transfer  (Nasr  et.  al.,  1997),  and  extends  recent 
work  reported  in  (Han  et  al.,  2000)  about  Korean 
to  English  transfer  in  particular.  Whereas  Han  et 
al.  focus  on  high  quality  domain-specific  transla¬ 
tion  using  handcrafted  transfer  rules,  in  this  work 
we  instead  focus  on  automating  the  acquisition  of 
such  rules. 

Our  approach  can  be  considered  a  generaliza¬ 
tion  of  syntactic  approaches  to  example-based 
machine  translation  (EBMT)  such  as  (Nagao, 
1984;  Sato  and  Nagao,  1990;  Maruyama  and 
Watanabe,  1992).  While  such  approaches  use 
syntactic  transfer  examples  during  the  actual 


transfer  of  source  parses,  our  approach  instead 
uses  syntactic  transfer  examples  to  induce  general 
transfer  rules  that  can  be  compiled  into  a  transfer 
dictionary  for  use  in  the  actual  translation  process. 

Our  approach  is  similar  to  the  recent  work  of 
(Meyers  et  al.,  1998)  where  transfer  rules  are  also 
derived  after  aligning  the  source  and  target  nodes 
of  corresponding  parses.  However,  it  also  differs 
from  (Meyers  et  al.,  1998)  in  several  important 
points.  The  first  difference  concerns  the  content 
of  parses  and  the  resulting  transfer  rules;  in  (Mey¬ 
ers  et  al.,  1998),  parses  contain  only  lexical  labels 
and  syntactic  roles  (as  arc  labels),  while  our  ap¬ 
proach  uses  parses  containing  lexical  labels,  syn¬ 
tactic  roles,  and  any  other  syntactic  information 
provided  by  parsers  (tense,  number,  person,  etc.). 
The  second  difference  concerns  the  node  align¬ 
ment;  in  (Meyers  et  al.,  1998),  the  alignment  of 
source  and  target  nodes  is  designed  in  a  way  that 
preserves  node  dominancy  in  the  source  and  tar¬ 
get  parses,  while  our  approach  does  not  have  such 
restriction.  One  of  the  reasons  for  this  difference 
is  due  to  the  different  language  pairs  under  study; 
(Meyers  et  al.,  1998)  deals  with  two  languages 
that  are  closely  related  syntactically  (Spanish  and 
English)  while  we  are  dealing  with  languages  that 
syntactically  are  quite  divergent,  Korean  and  En¬ 
glish  (Dorr,  1994).  The  third  difference  is  in  the 
process  of  identification  of  transfer  rules  candi¬ 
dates;  in  (Meyers  et  al.,  1998),  the  identification 
is  done  by  using  the  exact  tree  fragments  in  the 
source  and  target  parse  that  are  delimited  by  the 
alignment,  while  we  use  all  source  and  target  tree 
sub-patterns  matching  a  subset  of  the  parse  fea¬ 
tures  that  satisfy  a  customizable  set  of  alignment 
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constraints  and  attribute  constraints.  The  fourth 
third  difference  is  in  the  level  of  abstraction  of 
transferrules  candidates;  in  (Meyers  et  al.,  1998), 
the  source  and  target  patterns  of  each  transfer  rule 
are  fully  lexicalized  (except  possibly  the  terminal 
nodes),  while  in  our  approach  the  nodes  of  trans¬ 
fer  rules  do  not  have  to  be  lexicalized. 

Section  2  describes  our  approach  to  trans¬ 
fer  rules  induction  and  its  integration  with  data 
preparation  and  evaluation.  Section  3  describes 
the  data  preparation  process  and  resulting  data. 
Section  4  describes  the  transfer  induction  process 
in  detail.  Section  5  describes  the  results  of  our  ini¬ 
tial  evaluation.  Finally,  Section  6  concludes  with 
a  discussion  of  future  directions. 

2  Overall  Approach 

In  its  most  general  form,  our  approach  to  transfer 
rules  induction  includes  three  different  processes, 
data  preparation,  transfer  rule  induction  and  eval¬ 
uation.  An  overview  of  each  process  is  provided 
below;  further  details  are  provided  in  subsequent 
sections. 

The  data  preparation  process  creates  the  fol¬ 
lowing  resources  from  the  bi-texts: 

•  A  training  set  and  a  test  set  of  source  and 
target  parses  for  the  bi-texts,  post-processed 
into  a  syntactic  dependency  representation. 

•  A  baseline  transfer  dictionary,  which  may  in¬ 
clude  (depending  upon  availability)  lexical 
transfer  rules  extracted  from  the  bi-texts  us¬ 
ing  statistical  methods,  lexical  transfer  rules 
from  existing  bilingual  dictionaries,  and/or 
handcrafted  lexico-structural  transfer  rules. 

The  transfer  induction  process  induces  lexico- 
structural  transfer  rules  from  the  training  set  of 
corresponding  source  and  target  parses  that,  when 
added  to  the  baseline  transfer  dictionary,  produce 
transferred  parses  that  are  closer  to  the  corre¬ 
sponding  target  parses.  The  transfer  induction 
process  has  the  following  steps: 

•  Nodes  of  the  corresponding  source  and  tar¬ 
get  parses  are  aligned  using  the  baseline 
transfer  dictionary  and  some  heuristics  based 
on  the  similarity  of  part-of-speech  and  syn¬ 
tactic  context. 


•  Transfer  rule  candidates  are  generated  based 
on  the  sub-patterns  that  contain  the  corre¬ 
sponding  aligned  nodes  in  the  source  and  tar¬ 
get  parses. 

•  The  transfer  rule  candidates  are  ordered 
based  on  their  likelihood  ratios. 

•  The  transfer  rule  candidates  are  filtered,  one 
at  a  time,  in  the  order  of  the  likelihood  ra¬ 
tios,  by  removing  those  rule  candidates  that 
do  not  produce  an  overall  improvement  in 
the  accuracy  of  the  transferred  parses. 

The  evaluation  process  has  the  following  steps: 

•  Both  the  baseline  transfer  dictionary  and  the 
induced  transfer  dictionary  (i.e.,  the  baseline 
transfer  dictionary  augmented  with  the  in¬ 
duced  transfer  rules)  are  applied  to  the  test 
set  in  order  to  produce  two  sets  of  transferred 
parses,  the  baseline  set  and  the  (hopefully) 
improved  induced  set.  For  each  set,  the  dif¬ 
ferences  between  the  transferred  parses  and 
target  parses  are  measured,  and  the  improve¬ 
ment  in  tree  accuracy  is  calculated. 

•  After  performing  syntactic  realization  on  the 
baseline  set  and  the  induced  set  of  trans¬ 
ferred  parses,  the  differences  between  the 
resulting  translated  strings  and  the  target 
strings  are  measured,  and  the  improvement 
in  string  accuracy  is  calculated. 

•  For  a  subset  of  the  translated  strings,  human 
judgments  of  accuracy  and  grammaticality 
are  gathered,  and  the  correlations  between 
the  manual  and  automatic  scores  are  calcu¬ 
lated,  in  order  to  assess  the  meaningfulness 
of  the  automatic  measures. 

3  Data  Preparation 

3.1  Parsing  the  Bi-texts 

In  our  experiments  to  date,  we  have  used  a  cor¬ 
pus  consisting  of  a  Korean  dialog  of  4183  sen¬ 
tences  and  their  English  human  translations.  We 
ran  off-the-shelf  parsers  on  each  half  of  the  cor¬ 
pus,  namely  the  Korean  parser  developed  by  Yoon 
et  al.  (1997)  and  the  English  parser  developed  by 
Collins  (1997).  Neither  parser  was  trained  on  our 
corpus. 


We  automatically  converted  the  phrase  struc¬ 
ture  output  of  the  Collins  parser  into  the  syntac¬ 
tic  dependency  representation  used  by  our  syn¬ 
tactic  realizer,  RealPro  (Lavoie  and  Rambow, 
1997).  This  representation  is  based  on  the  deep- 
syntactic  structures  (DSyntS)  of  Meaning-Text 
Theory  (Mel’cuk,  1988).  The  important  features 
of  a  DSyntS  are  as  follows: 

•  a  DSyntS  is  an  unordered  tree  with  labeled 
nodes  and  labeled  arcs; 

•  a  DSyntS  is  lexicalized,  meaning  that  the 
nodes  are  labeled  with  lexemes  (uninllected 
words)  from  the  target  language; 

•  a  DSyntS  is  a  dependency  structure  and  not  a 
phrase-  structure  structure:  there  are  no  non¬ 
terminal  nodes,  and  all  nodes  are  labeled 
with  lexemes; 

•  a  DSyntS  is  a  syntactic  representation, 
meaning  that  the  arcs  of  the  tree  are  la¬ 
beled  with  syntactic  relations  such  as  SUB¬ 
JECT  (represented  in  DSyntSs  as  I),  rather 
than  conceptual  or  semantic  relations  such  as 
AGENT; 

•  a  DSyntS  is  a  deep  syntactic  representation, 
meaning  that  only  meaning-bearing  lexemes 
are  represented,  and  not  function  words. 

Since  the  output  of  the  Yoon  parser  is  quite  sim¬ 
ilar,  with  the  exception  of  its  treatment  of  syn¬ 
tactic  relations,  we  have  used  its  output  as  is. 
The  DSyntS  representations  for  two  correspond¬ 
ing  Korean'  and  English  sentences  are  illustrated 
in  Figure  1. 

In  examining  the  outputs  of  the  two  parsers 
on  our  corpus,  we  found  that  about  half  of  the 
parse  pairs  contained  incorrect  dependency  as¬ 
signments,  incomplete  lemmatization  or  incom¬ 
plete  parses.  To  reduce  the  impact  of  such  pars¬ 
ing  errors  in  our  initial  experiments,  we  have  pri¬ 
marily  focused  on  a  higher  quality  subset  of  1763 
sentence  pairs  that  were  selected  according  to  the 
following  criteria: 

•  Parse  pairs  where  the  source  or  target  parse 
contained  more  than  10  nodes  were  rejected, 

•  Korean  is  represented  in  romanized  format  in  this  paper. 


(51)  {1}  {Ci-To-Reul}  {Ta-Si}  {Po-Ra}. 

this  +  map-accusative  +  again  +  look-imp 

(Dl)  {po}  [class=vbma  ente={ra}]  ( 

si  {ci-to}  [class=nnin2  ppca={reul}]  { 
si  {i}  [class=ande] 

) 

si  {ta-si}  [class=adco2 ] 

) 

(52)  Look  at  the  map  again. 

(D2)  look  [class=verb  mood=imp]  ( 

attr  at  [class=preposition]  ( 

ii  map  [class=common_noun  article=def] 

) 

attr  again  [class=adverb] 

) 


Figure  1:  Syntactic  dependency  representations 
for  corresponding  Korean  and  English  sentences 

since  these  usually  contained  more  parse  er¬ 
rors  than  smaller  parses. 

•  Parse  pairs  where  the  source  or  target  parse 
contained  non-final  punctuation  were  re¬ 
jected;  this  criterion  was  based  on  our  ob¬ 
servation  that  in  most  such  cases,  the  source 
or  target  parses  contained  only  a  fragment 
of  the  original  sentence  content  (i.e.,  one  or 
both  parsers  only  parsed  what  was  on  one 
side  of  an  intra-sentential  punctuation  mark). 

We  divided  this  higher  quality  subset  into  train¬ 
ing  and  test  sets  by  randomly  choosing  50%  of 
the  1763  higher  quality  parse  pairs  (described  in 
Section  3.1)  for  inclusion  in  the  training  set,  re¬ 
serving  the  remaining  50%  for  the  test  set.  The 
average  numbers  of  parse  nodes  in  the  training  set 
and  test  set  were  respectively  6.91  and  6.1 1  nodes. 

3.2  Creating  the  Baseline  Transfer 
Dictionary 

In  the  general  case,  any  available  bilingual  dic¬ 
tionaries  can  be  combined  to  create  the  base¬ 
line  transfer  dictionary.  These  dictionaries  may 
include  lexical  transfer  dictionaries  extracted 
from  the  bi-texts  using  statistical  methods,  exist¬ 
ing  bilingual  dictionaries,  or  handcrafted  lexico- 
structural  transfer  dictionaries.  If  probabilistic  in¬ 
formation  is  not  already  associated  with  the  lexi¬ 
cal  entries,  log  likelihood  ratios  can  be  computed 
and  added  to  these  entries  based  on  the  occur¬ 
rences  of  these  lexical  items  in  the  parse  pairs. 

In  our  initial  experiments,  we  decided  to  focus 
on  the  scenario  where  the  baseline  transfer  die- 


@ KOREAN: 

{po}  [class=vbma]  ( 
si  $X  [ppca= { reul } ] 

) 

@ENGLISH: 

look  [class=verb]  ( 

attr  at  [class=preposition]  ( 
ii  $X 

) 

) 

@-2xL0G_LIKELIH00D :  12.77 


Figure  2:  Transfer  rule  for  English  lexiealization 
and  preposition  insertion 


@ KOREAN: 

$X  [class=vbma  ente={ra}] 
@ENGLISH: 

$X  [class=verb  mood=imp] 
@-2xL0G_LIKELIH00D :  33.37 


Figure  3:  Transfer  rule  for  imperative  forms 

tionary  is  ereated  from  lexieal  transfer  entries  ex- 
traeted  from  the  bi-texts  using  statistieal  methods. 
To  simulate  this  seenario,  we  ereated  our  baseline 
transfer  dietionary  by  taking  the  lexieo-syntaetie 
transfer  dietionary  developed  by  Han  et  al.  (2000) 
for  this  eorpus  and  removing  the  (more  general) 
rules  that  were  not  fully  lexiealized.  Starting  with 
this  purely  lexieal  baseline  transfer  dietionary  en¬ 
abled  us  to  examine  whether  these  more  general 
rules  eould  be  diseovered  through  induetion. 

4  Transfer  Rule  Induction 

The  indueed  lexieo-struetural  transfer  rules  are 
represented  in  a  formalism  similar  to  the  one  de- 
seribed  in  Nasr  et  al.  (1997),  and  extended  to  also 
inelude  log  likelihood  ratios.  Figures  2  and  3 
illustrate  two  entry  samples  that  ean  be  used  to 
transfer  a  Korean  syntaetie  representation  for  ci- 
to-reul  po-ra  to  an  English  syntaetie  representa¬ 
tion  for  look  at  the  map.  The  hrst  rule  lexiealizes 
the  English  predieate  and  inserts  the  eorrespond- 
ing  preposition  while  the  seeond  rule  inserts  the 
English  imperative  attribute.  This  formalism  uses 
notation  similar  to  the  syntaetie  dependeney  nota¬ 
tion  shown  in  Figure  1,  augmented  with  variable 
arguments  prefixed  with  $  eharaeters. 


4.1  Aligning  the  Parse  Nodes 

To  align  the  nodes  in  the  souree  and  target  parse 
trees,  we  devised  a  new  dynamie  programming 
alignment  algorithm  that  performs  a  top-down, 
bidireetional  beam  seareh  for  the  least  eost  map¬ 
ping  between  these  nodes.  The  algorithm  is  pa¬ 
rameterized  by  the  eosts  of  (1)  aligning  two  nodes 
whose  lexemes  are  not  found  in  the  baseline  trans¬ 
fer  dietionary;  (2)  aligning  two  nodes  with  dif¬ 
fering  parts  of  speeeh;  (3)  deleting  or  inserting  a 
node  in  the  souree  or  target  tree;  and  (4)  aligning 
two  nodes  whose  relative  loeations  differ. 

To  determine  an  appropriate  part  of  speeeh  eost 
measure,  we  first  extraeted  a  small  set  of  parse 
pairs  that  eould  be  reliably  aligned  using  lexieal 
matehing  alone,  and  then  based  the  eost  measure 
on  the  eo-oeeurrenee  eounts  of  the  observed  parts 
of  speeeh  pairings.  The  remaining  eosts  were  set 
by  hand. 

As  a  result  of  the  alignment  proeess,  alignment 
id  attributes  (aid)  are  added  to  the  nodes  of  the 
parse  pairs.  Some  nodes  may  be  in  alignment 
with  no  other  node,  sueh  as  English  prepositions 
not  found  in  the  Korean  DSyntS. 

4.2  Generating  Rule  Candidates 

Candidate  transfer  rules  are  generated  using  three 
data  sourees: 

•  the  training  set  of  aligned  souree  and  target 
parses  resulting  from  the  alignment  proeess; 

•  a  set  of  alignment  eonstraints  whieh  identify 
the  subtrees  of  interest  in  the  aligned  souree 
and  target  parses  (Seetion  4.2.1); 

•  a  set  of  attribute  eonstraints  whieh  determine 
what  parts  of  the  aligned  subtrees  to  inelude 
in  the  transfer  rule  eandidates’  souree  and 
target  patterns  (Seetion  4.2.2). 

The  alignment  and  attribute  eonstraints  are  nee- 
essary  to  keep  the  set  of  eandidate  transfer  rules 
manageable  in  size. 

4,2,1  Alignment  constraints 

Figure  4  shows  an  example  alignment  eonstraint. 
This  eonstraint,  whieh  matehes  the  struetural  pat¬ 
terns  of  the  transfer  rule  illustrated  in  Figure  2, 
uses  the  aid  alignment  attribute  to  indieate  that 


@ KOREAN: 

$X1  [aid=$l]  ( 

$R1  $X2  [aid=$2] 

) 

@ENGLISH: 

$Y1  [aid=$l]  ( 

$R2  $Y2  ( 

$R3  $Y3  [aid=$2] 

) 

) 


Figure  4:  Alignment  eonstraint 


in  a  Korean  and  English  parse  pair,  any  souree 
and  target  sub-trees  matehing  this  alignment  eon¬ 
straint  (where  $X1  and  $Y1  are  aligned  or  have 
the  same  attribute  aid  values  and  where  $X2  and 
$Y3  are  aligned)  ean  be  used  as  a  point  of  depar¬ 
ture  for  generating  transfer  rule  eandidates.  We 
suggest  that  alignment  eonstraints  sueh  as  this  one 
ean  be  used  to  define  most  of  the  possible  syntae- 
tie  divergenees  between  languages  (Dorr,  1994), 
and  that  only  a  handful  of  them  are  neeessary  for 
two  given  languages  (we  have  identified  1 1  gen¬ 
eral  alignmenf  eonsfrainfs  neeessary  for  Korean  fo 
English  Iransfer  so  far). 

4.2.2  Attribute  constraints 

Attribute  eonstraints  are  used  to  limit  the  spaee 
of  possible  transfer  rule  eandidates  that  ean  be 
generated  from  the  sub-trees  satisfying  the  align¬ 
ment  eonstraints.  Candidate  transfer  rules  must 
satisfy  all  of  the  attribute  eonstraints.  Attribute 
eonstraints  ean  be  divided  into  two  types: 

•  independent  attribute  constraints,  whose 
seope  eovers  only  one  part  of  a  eandidate 
transfer  rule  and  whieh  are  the  same  for  the 
souree  and  target  parts; 

•  concurrent  attribute  constraints,  whose 
seope  extends  to  both  the  souree  and  target 
parts  of  a  eandidate  transfer  rule. 

The  examples  of  an  independent  attribute  eon¬ 
straint  and  of  a  eoneurrent  attribute  eonstraint  are 
given  in  Eigure  5  and  Eigure  6  respeetively.  As 
with  the  alignment  eonstraints,  we  suggest  that  a 
relatively  small  number  of  attribute  eonstraints  is 
neeessary  to  generate  most  of  the  desired  rules  for 
a  given  language  pair. 


Each  node  of  a  candidate  transfer  rule  must  have  its  relation 
attribute  (relationship  with  its  governor)  specified  if  it  is  an 
internal  node,  otherwise  this  relation  must  not  be  specified: 


e.g. 

0  $X1  (  $R  $X2  ) 


Eigure  5 :  Independent  attribute  eonstraint 

In  a  candidate  transfer  rule,  inclusion  of  the  lexemes  of  two 
aligned  nodes  must  be  done  concurrently: 

e.g. 

$x 

[aid=$l] 

and 

$Y 

[aid=$l ] 

e.g. 

0 

[aid=$l] 

and 

0 

[aid=$l] 

Eigure  6:  Coneurrent  attribute  eonstraint 


4.3  Ordering  Rule  Candidates 

In  the  next  step,  transfer  rule  eandidates  are  or¬ 
dered  as  follows:  first,  by  their  log  likelihood  ra¬ 
tios  (Manning  and  Sehutze,  1999:  172-175);  see- 
ond,  any  transfer  rule  eandidates  with  the  same 
log  likelihood  ratio  are  ordered  by  their  speei- 
fieity. 

4.3.1  Rule  ordering  by  log  likelihood  ratio 

We  ealeulate  the  log  likelihood  ratio,  log  A,  ap¬ 
plied  to  a  transfer  rule  eandidate  as  indieated  in 
Eigure  7.  Note  that  log  A  is  a  negative  value, 
and  following  (Manning  and  Sehutze,  1999),  we 
assign  -2  log  A  to  the  transfer  rule.  Note  also 
that  in  the  definitions  of  Ci,  C2,  and  C12  we  are 
eurrently  only  eonsidering  one  oeeurrenee  or  eo- 
oeeurrenee  of  the  souree  and/or  target  patterns  per 
parse  pair,  while  in  general  there  eould  be  more 
than  one;  in  our  initial  experiments  these  defini¬ 
tions  have  suffieed. 

4.3.2  Rule  ordering  by  specificity 

If  two  or  more  eandidate  transfer  rules  have  the 
same  log  likelihood  ratio,  ties  are  broken  by  a 
speeifieity  heuristie,  with  the  result  that  more  gen¬ 
eral  rules  are  ordered  ahead  of  more  speeifie  ones. 
The  speeifieity  of  a  rule  is  defined  to  be  the  fol¬ 
lowing  sum:  the  number  of  attributes  found  in 
the  souree  and  target  patterns,  plus  1  for  eaeh  for 


log  A  = 

log L{Ci2,Ci,p)  +  \ogL{C2  -  Ci2,N  -  Ci,p) 

-  log  L(Ci2  ,  Cl ,  pi )  -  log  L  (Ca  -  Cia ,  iV  -  Cl ,  P2 ) 

where,  not  counting  attributes  aid, 

•  Cl  =  number  of  source  parses  containing  at  least  one 
occurrence  of  C’s  source  pattern 

•  Ca  =  number  of  target  parses  containing  at  least  one 
occurrence  of  C’s  target  pattern 

•  Ci2  =  number  of  source  and  target  parse  pairs  contain¬ 
ing  at  least  one  co-occurrence  of  C’s  source  pattern 
and  C’s  target  pattern  satisfying  the  alignment  con¬ 
straints 

•  N  =  number  of  source  and  target  parse  pairs 

•  P^Ca/iV; 

•  Pl^Cl2lCl- 

•  P2  =  (C2-Ci2)/(iV-Ci); 

•  L(fc,  n,  x)  =  x*(l  — 


Figure  7:  Log  likelihood  ratios  for  transfer  rule 
candidates 

each  lexeme  attribute  and  for  each  dependency  re¬ 
lationship.  In  our  initial  experiments,  this  simple 
heuristic  has  been  satisfactory. 

4.4  Filtering  Rule  Candidates 

Once  the  candidate  transfer  rules  have  been  or¬ 
dered,  error-driven  hltering  is  used  to  select  those 
that  yield  improvements  over  the  baseline  trans¬ 
fer  dictionary.  The  algorithm  works  as  follows. 
First,  in  the  initialization  step,  the  set  of  accepted 
transfer  rules  is  set  to  just  those  appearing  in  the 
baseline  transfer  dictionary,  and  the  current  er¬ 
ror  rate  is  established  by  applying  these  transfer 
rules  to  ah  the  source  structures  and  calculating 
the  overall  difference  between  the  resulting  trans¬ 
ferred  structures  and  the  target  parses.  Then,  in  a 
single  pass  through  the  ordered  list  of  candidates, 
each  transfer  rule  candidate  is  tested  to  see  if  it 
reduces  the  error  rate.  During  each  iteration,  the 
candidate  transfer  rule  is  provisionally  added  to 
the  current  set  of  accepted  rules  and  the  updated 
set  is  applied  to  ah  the  source  structures.  If  the 
overall  difference  between  the  transferred  struc¬ 
tures  and  the  target  parses  is  lower  than  the  cur¬ 
rent  error  rate,  then  the  candidate  is  accepted  and 


©KOREAN: 

{po}  [class=vbma  ente={ra}]  ( 
si  $X  [ppca= { reul } ] 

) 

©ENGLISH : 

look  [class=verb  mood=imp]  ( 
attr  at  [class=preposition]  ( 
ii  $X 

) 

) 

©-2xL0G_LIKELIH00D :  11.40 


Figure  8:  Transfer  rule  for  English  imperative 
with  lexicalization  and  preposition  insertion 


the  current  error  rate  is  updated;  otherwise,  the 
candidate  is  rejected  and  removed  from  the  cur¬ 
rent  set. 

4.5  Discussion  of  Induced  Rules 

Experimentation  with  the  training  set  of  882  parse 
pairs  described  in  Section  3.1  produced  12467 
source  and  target  sub-tree  pairs  using  the  align¬ 
ment  constraints,  from  which  20569  transfer  rules 
candidate  were  generated  and  7565  were  accepted 
after  hltering.  We  expect  that  the  number  of 
accepted  rules  per  parse  pair  will  decrease  with 
larger  training  sets,  though  this  remains  to  be  ver- 
ihed. 

The  rule  illustrated  in  Eigure  3  was  accepted  as 
the  65th  best  transfer  rule  with  a  log  likelihood 
ratio  of  33.37,  and  the  rule  illustrated  in  Eigure  2 
was  accepted  as  the  1 89th  best  transfer  rule  can¬ 
didate  with  a  log  likelihood  ratio  of  12.77.  An  ex¬ 
ample  of  a  candidate  transfer  rule  that  was  not  ac¬ 
cepted  is  the  one  that  combines  the  features  of  the 
two  rules  mentioned  above,  illustrated  in  Eigure  8. 
This  transfer  rule  candidate  had  a  lower  log  like¬ 
lihood  ratio  of  1 1.40;  consequently,  it  is  only  con¬ 
sidered  after  the  two  rules  mentioned  above,  and 
since  it  provides  no  further  improvement  upon 
these  two  rules,  it  is  hltered  out. 

In  an  informal  inspection  of  the  top  100  ac¬ 
cepted  transfer  rules,  we  found  that  most  of  them 
appear  to  be  fairly  general  rules  that  would  nor¬ 
mally  be  found  in  a  general  syntactic -based  trans¬ 
fer  dictionary.  In  looking  at  the  remaining  rules, 
we  found  that  the  rules  tended  to  become  increas¬ 
ingly  corpus-specihc. 


5  Initial  Evaluation 
5,1  Results 

In  an  initial  evaluation  of  our  approach,  we  ap¬ 
plied  both  the  baseline  transfer  dictionary  and 
the  induced  transfer  dictionary  (i.e.,  the  baseline 
transfer  dictionary  augmented  with  the  transfer 
rules  induced  from  the  training  set)  to  the  test  half 
of  the  1763  higher  quality  parse  pairs  described  in 
Section  3.1,  in  order  to  produce  two  sets  of  trans¬ 
ferred  parses,  the  baseline  set  and  the  induced  set. 
For  each  set,  we  then  calculated  tree  accuracy  re¬ 
call  and  precision  measures  as  follows: 

Tree  accuracy  recall  The  tree  accuracy  recall 
for  a  transferred  parse  and  a  correspond¬ 
ing  target  parse  is  determined  the  by  67/ Rq, 
where  67  is  the  total  number  of  features  (at¬ 
tributes,  lexemes  and  dependency  relation¬ 
ships)  that  are  found  in  both  the  nodes  of 
the  transferred  parse  and  in  the  correspond¬ 
ing  nodes  in  the  target  parse,  and  Rq  is  the 
total  number  of  features  found  in  the  nodes 
of  the  target  parse.  The  correspondence  be¬ 
tween  the  nodes  of  the  transferred  parse  and 
the  nodes  of  the  target  parse  is  determined 
with  alignment  information  obtained  using 
the  technique  described  in  Section  4.1. 

Tree  accuracy  precision  The  tree  accuracy  pre¬ 
cision  for  a  transferred  parse  and  a  corre¬ 
sponding  target  parse  is  determined  the  by 
C/Rt,  where  67  is  the  total  number  of  fea¬ 
tures  (attributes,  lexemes  and  dependency 
relationships)  that  are  found  in  both  the 
nodes  of  the  transferred  parse  and  in  the  cor¬ 
responding  nodes  in  the  target  parse,  and  Rt 
is  the  total  number  of  features  found  in  the 
nodes  of  the  transferred  parse. 

Table  1  shows  the  tree  accuracy  results,  where 
the  f-score  is  equally  weighted  between  recall  and 
precision.  The  results  illustrated  in  Table  1  indi¬ 
cate  that  the  transferred  parses  obtained  using  in¬ 
duction  were  moderately  more  similar  to  the  tar¬ 
get  parses  than  the  transferred  parses  obtained  us¬ 
ing  the  baseline  transfer,  with  about  15  percent 
improvement  in  the  f-score. 


Recall 

Precision 

F-Score 

Baseline 

Inducfion 

37.77 

55.35 

46.81 

58.20 

41.18 

55.82 

Table  1 :  Tree  accuracy  results 


5.2  Discussion 

At  the  time  of  writing,  the  improvements  in  tree 
accuracy  do  not  yet  appear  to  yield  apprecia¬ 
ble  improvements  in  realization  results.  While 
our  syntactic  realizer,  RealPro,  does  produce  rea¬ 
sonable  surface  strings  from  the  target  depen¬ 
dency  trees,  despite  occasional  errors  in  parsing 
the  target  strings  and  converting  the  phrase  struc¬ 
ture  trees  to  dependency  trees,  it  appears  that  the 
tree  accuracy  levels  for  the  transferred  parses  will 
need  to  be  higher  on  average  before  the  improve¬ 
ments  in  tree  accuracy  become  consistently  visi¬ 
ble  in  the  realization  results.  At  present,  the  fol¬ 
lowing  three  problems  represent  the  most  impor¬ 
tant  obstacles  we  have  identified  to  achieving  bet¬ 
ter  end-to-end  results: 

•  Since  many  of  the  test  sentences  require 
transfer  rules  for  which  there  are  no  similar 
cases  in  the  set  of  training  sentences,  it  ap¬ 
pears  that  the  relatively  small  size  of  our  cor¬ 
pus  is  a  significant  barrier  to  better  results. 

•  Some  performance  problems  with  the  cur¬ 
rent  implementation  have  forced  us  to  make 
use  of  a  perhaps  overly  strict  set  of  alignment 
and  attribute  constraints.  With  an  improved 
implementation,  it  may  be  possible  to  find 
more  valuable  rules  from  fhe  same  fraining 
dafa. 

•  A  more  refined  freafmenf  of  rule  conflicls  is 
needed  in  order  fo  allow  multiple  rules  fo 
access  overlapping  confexfs,  while  avoiding 
fhe  infroducfion  of  mulfiple  franslafions  of 
fhe  same  confenf  in  cerfain  cases. 

6  Conclusion  and  Future  Directions 

In  fhis  paper  we  have  presenfed  a  novel  approach 
fo  fransfer  rule  inducfion  based  on  synlacfic  pal- 
fern  co-occurrence  in  parsed  bi-lexls.  In  an  inilial 
evaluation  on  a  relatively  small  corpus,  we  have 


shown  that  the  induced  syntactic  transfer  rules 
from  Korean  to  English  lead  to  a  modest  increase 
in  the  accuracy  of  transferred  parses  when  com¬ 
pared  to  the  target  parses.  In  future  work,  we 
hope  to  demonstrate  that  a  combination  of  consid¬ 
ering  a  larger  set  of  transfer  rule  candiates,  refin¬ 
ing  our  treatment  of  rule  conflicts,  and  making  use 
of  more  training  data  will  lead  to  further  improve¬ 
ments  in  tree  accuracy,  and,  following  syntactic 
realization,  will  yield  to  significant  improvements 
in  end-to-end  results. 
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